Implementation of a mixed-reality flight simulator: blending real and virtual with a video-see-through head-mounted display

Conventional flight simulators usually include a complex and expensive outside vision projection system. Especially scenarios where the helicopter pilots look far to the side or through the windows near the pedals require a large projection dome to provide an image of the outside world. Additionally, simulators used for research need to be highly customizable: For rapid prototyping of new flight deck designs, the cockpit mockup must be adaptable enough to change the appearance and arrangement of its elements. The recent technological advancements of head-mounted displays (HMDs) offer many new ways to create a simulator that fulfills the stated requirements at moderate cost. A non-see-through HMD can immerse the pilots into a computer-generated cockpit with "unrestricted" virtual out-the-window view. The downwards view is even better than with dome projections. Such a fully virtual approach, however, requires complex finger-tracking and haptic feedback solutions to enable the user to interact with the cockpit. By contrast, a video-see-through HMD allows us to selectively combine a highly customizable virtual world with a video-stream of the real surroundings. One can, for instance, show the pilot's hands and relevant parts of the physical flight deck mockup, enriched with virtual elements and virtual out-the-window view. In such a mixed setup, the pilots can naturally and directly interact with conventional input devices in an otherwise virtual environment. The paper presents our implementation of a mixed reality simulator with the Varjo XR-3 video-see-through HMD. We assess different variants, discuss implementation details like real-to-virtual-world-alignment, and explain the major challenges of such setups

Virtual Cockpit Instruments - How Head-Worn Displays Can Enhance the Obstacle Awareness of Helicopter Pilots

The rise of augmented reality glasses and related technologies offers new possibilities for the human-machine interface design of future aircraft. Today, head-worn displays (HWDs) are mainly used by military pilots, for instance by helicopter crews for low-visibility operations close to ground and obstacles. Nevertheless, recent technological advances in this area allow the prediction that these systems could become available for more pilots in the future. This article presents a concept how state-of-the-art HWD symbology can be expanded to get even more out of the advantages of this technology. With so-called "virtual cockpit instruments" (VCIs), an HWD can show information, which is conventionally rendered on panel-mounted displays. These VCIs can be imagined as virtual display screens, which can be positioned freely around the pilot. Their major benefit is that they create many new options for the design of a flexible, situation-adaptive cockpit environment. This article introduces the general concept and presents several options how such an approach can be put into practice. Here, the concept is applied to helicopter operations in offshore windparks. We implemented a VCI-adapted obstacle awareness display and assessed a set of positioning variants for the new VCI. Two simulator studies -- with 11 and 7 participants -- provide interesting insights on the realization of this concept. In addition to high subjective ratings, the VCI significantly increased the pilot's head-up, eyes-out time -- an important measure for challenging maneuvers close to obstacles. Overall, this article illustrates a promising concept for the human-machine interface design of future cockpits and discusses its potentials and limitations

Bringing a colored head-down display symbology heads up: Display fidelity review of a low-cost see-through HMD

Within the U.S.-German project agreement on advanced technology for rotorcraft (PA ATR) DLR conducts research on pilot aids for helicopter operations in DVE. For example, an HMD can provide visual cues to the pilot while flying eyes-out. With the HoloLens 2 as a low-budget device, we can rapidly prototype new symbology and make use of its full-color waveguide display. As a first step, this paper aims to investigate how the color of DLR's symbology needs to be adapted when shown on the HoloLens 2 and what kind of information is beneficial when presented on an HMD or should remain on a head-down display

Entwicklung eines konfigurierbaren Flugtaxi-Simulators mit Hilfe einer kopfgetragenen Anzeige mit Video-Durchsicht - Ein Vergleich verschiedener Mixed-Reality-Ansätze

Um zukünftige Flugtaxi-Konzepte frühzeitig aus Passagiersicht bewerten zu können, wird ein immersiver und gleichzeitig konfigurierbarer Kabinensimulator benötigt. Dieser Vortrag zeigt wie moderne Mixed-Reality-Technologien bei der Umsetzung eines solchen Simulators helfen können. So kann zum Beispiel mit Hilfe einer Video-Durchsicht-Brille das reale Kabinenmockup flexibel um virtuelle Objekte erweitert werden. Die durchgeführte Probandenstudie zeigt die Vorteile und Grenzen solcher Ansätze

Experiencing Urban Air Mobility: How Passengers evaluate a simulated flight with an Air Taxi

For the successful development and implementation of novel concepts and technology, the acceptance of potential users is crucial. Therefore, within the project HorizonUAM, we investigated passengers' acceptance of air taxis. One challenge is that not many people have real experiences with urban air mobility (UAM) at the moment and thus requirements formulated by potential users refer to rather abstract concepts. To allow participants to gain realistic impressions of UAM concepts, a Mixed Reality Air Taxi Simulator was set up. It allows participants to experience an inner-city business shuttle flight. A study with 30 participants assessed the information needs and the influence of another person on board on wellbeing in nominal situations (experiment 1) as well as one non-nominal situation (experiment 2). For the latter, participants experienced a re-routing of the flight due to an unavailability of landing sites at the vertidrome. During and after the flights, participants answered questionnaires and extensive interviews were conducted. The study produced first empirical data on relevant factors regarding interaction, information needs and comfort within an air taxi. The findings show that passengers want to be informed about intentions of the vehicle. The presence of a steward on board is not necessary but can increase wellbeing especially during non-nominal situations.Comment: 16 pages, 12 figures, 8 table

Assessment of Air Taxi Passenger Acceptance – Implementation and Initial Evaluation of a Mixed Reality Simulator

Urban air mobility paves new ways for public transport modalities like air taxis. To shape the operational concepts and the design of these air taxis, the acceptance and requirements of future passengers should be considered even in early stages of the development process. Within the HorizonUAM project, it is of interest to understand the effects of social presence, e.g. through on-board service personnel, on perceived comfort. These results help to develop concepts, how fully or mainly autonomous operations can be achieved that are accepted by users. As part of HorizonUAM, an air taxi simulator was implemented to assess the aforementioned passenger acceptance with regard to defined key criteria like presence of on-board personnel or amount of presented information. To create an experience that is as close to a real air taxi flight as possible, recent advancements in head-mounted display technology facilitated the creation of a mixed reality simulation. Mixed reality combines advantages of full-flight simulators like human collaboration and use of analog instruments with those of virtual reality simulators that are cost-effective, highly immersive devices for rapid prototyping. This paper reports on the work conducted to define and design such a mixed reality simulation to investigate factors of passenger acceptance. First, functional requirements that were defined for a first study of passenger interaction and comfort are presented, as well as the technical requirements that were derived. Additionally, mixed reality technology should be used for the novel simulator to enable high flexibility. Second, as numerable different setups of mixed reality were possible with state-of-the-art technology, four different setups were compared in a pre-study against each other. The empirical results with regards to experienced immersion are presented. Finally, the conclusion from this study regarding the final setup for the mixed reality simulation are presented

Systematic reviews of complementary therapies - an annotated bibliography. Part 1: Acupuncture

Background Complementary therapies are widespread but controversial. We aim to provide a comprehensive collection and a summary of systematic reviews of clinical trials in three major complementary therapies (acupuncture, herbal medicine, homeopathy). This article is dealing with acupuncture. Potentially relevant reviews were searched through the register of the Cochrane Complementary Medicine Field, the Cochrane Library, Medline, and bibliographies of articles and books. To be included articles had to review prospective clinical trials of acupuncture; had to describe review methods explicitly; had to be published; and had to focus on treatment effects. Information on conditions, interventions, methods, results and conclusions was extracted using a pretested form and summarized descriptively. Results From a total of 48 potentially relevant reviews preselected in a screeening process 39 met the inclusion criteria. 22 were on various pain syndromes or rheumatic diseases. Other topics addressed by more than one review were addiction, nausea, asthma and tinnitus. Almost unanimously the reviews state that acupuncture trials include too few patients. Often included trials are heterogeneous regarding patients, interventions and outcome measures, are considered to have insufficient quality and contradictory results. Convincing evidence is available only for postoperative nausea, for which acupuncture appears to be of benefit, and smoking cessation, where acupuncture is no more effective than sham acupuncture. Conclusions A large number of systematic reviews on acupuncture exists. What is most obvious from these reviews is the need for (the funding of) well-designed, larger clinical trials

iUSIM - Institutsübergreifende, modulare Urban Mobility Simulationsinfrastruktur

Generische Simulationsmodelle sind essenziell für die Bewertung neuer Konzepte, Verfahren und Methoden im Bereich Luftfahrt und Verkehr und stellen somit einen wesentlichen Bestandteil unserer Forschung dar. Die Institute für Flugführung (FL), Verkehrssystemtechnik (TS) und Softwaretechnologie (SC) verfügen jeweils über fundierte Kenntnisse und Expertisen hinsichtlich verschiedener Simulationstechnologien und -möglichkeiten. Diese wurden im Rahmen des Projektes zusammengetragen und ausgetauscht. Anhand eines Rettungsszenarios steht insbesondere die Verbindung der einzelnen Simulatoren von FL, TS, SC im Fokus dieser Arbeit

Can Urban Air Mobility become reality? Opportunities, challenges and selected research results

Urban Air Mobility (UAM) is a new air transportation system for passengers and cargo in urban environments, enabled by new technologies and integrated into multimodal transportation systems. The vision of UAM comprises the mass use in urban and suburban environments, complementing existing transportation systems and contributing to the decarbonization of the transport sector. Initial attempts to create a market for urban air transportation in the last century failed due to lack of profitability and community acceptance. Technological advances in numerous fields over the past few decades have led to a renewed interest in urban air transportation. UAM is expected to benefit users and to also have a positive impact on the economy by creating new markets and employment opportunities for manufacturing and operation of UAM vehicles and the construction of related ground infrastructure. However, there are also concerns about noise, safety and security, privacy and environmental impacts. Therefore, the UAM system needs to be designed carefully to become safe, affordable, accessible, environmentally friendly, economically viable and thus sustainable. This paper provides an overview of selected key research topics related to UAM and how the German Aerospace Center (DLR) contributed to this research in the project "HorizonUAM - Urban Air Mobility Research at the German Aerospace Center (DLR)". Selected research results that support the realization of the UAM vision are briefly presented.Comment: 20 pages, 7 figures, project HorizonUA